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This paper's original form was a B.S. thesis supervised by P.H. Winston* 

* * * 

Work reported herein wa? supported by the Warren Mcculloch Laboratory, 
an M.I*T. nesearch program sponsored by the Advanced Research Projects 
Agency of the Department of Defense under office of Naval Research 
contract number NOO014-7O-A-0362-0O02. 

Reproduction of this document, in whole or in part, is permitted for 
any purpose of the United States Government. 



TABLE OF CONTENTS 



Introduction ..5 

1.1 Computer vision 5 

1.2 Possible improvements to SEE 10 

1.3 SEEHORE: an extension of SEE 12 

1.4 A note on nomenclature ■*« * * * * * 14 

The basic SEODBE procedure 18 

2.1 Phase 1: generate Initial apllti and links - • 18 

2.2 Phase 2: generate other plausible splits ... 30 

2.3 Phase 3: link and conglomerate A3 

Producing Alternatives 51 

3.1 The nature of a complaint 51 

3.2 The alternative-list 51 

3.3 The processing of a complaint 56 

Some examples . . * . # • • * * » « 59 

A. 1 A detailed example 59 

4*2 Other examples 66 

Failures of SEEMORE and recommendations ■ • > • 68 



-5- 



1 Introduction 

1.1 Computer vision 

For the last several years, the Artificial Intelligence Croup of 
M.I.T. 's Project KAC has been seeking to develop more complex and 
powerful ways for computers to interact with their environment. One 
of the products of this research is a primitive scene-perceiving 
system, which is represented schematically in the flow diagram of 
figure 1-X- 

1.1. 1 Pre-processing 

The system begins with an actual scene, from which all informa- 
tion ultimately derives. The eye of the system is a television 
camera specifically designed to Interact with a computer. A program 
developed by Griffith [1] processes the output of the camera to 
produce a line drawing of the scene. This line drawing, appropriate- 
ly corrected and encoded, is used as the representation of the scene 
itself by all higher-level programs in the system. 

1.1.2 Processing of the line drawing 

Much of the following description of the MAC vision system is 
taken from Patrick H. Winston's Ph.D. thesis [2]. 
1.1.2,1 Classification of vertices 

A program written by H. W. Mahabala [3] classifies and labels the 
vertices in the scene according to the number of converging lines at 
each vertex and the angles between them. Figure 1-2 displays the 
vertex types which are recognized. Notice that pairs of Ts with 
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crossbars lying between colllncar uprights are found by Mahafrila'p 
program. These are called natching~Ts. 

The program then proceeds to create rumcv for all of the rt^* t lun» 
in the scene. Rigorously, "region" as used here simply refers to any 
maximal area in which one can oove from any point to any other point 
without crossing a line. For example, figure 1-3 hag eight regions* 
not counting the background. Various properties are calculated and 
stored for these regions- Among these are a list of the vertices and 
bordering regions which surround each region. 
1.1.2.2 The SEE procedure 

These results are then supplied to the program named SEE devel- 
oped by A. Guzman [A], This program conjectures about which regions 
belong to the same object*. For figure l-3» the end result of the 
program is the commentary; 

Body 1 consists of A B C 
Body 2 consists of D E F G H 
Surprisingly the program contains no explicit models for the objects 
it expects to see. it simply examines the vertices and uses the 
vertex classifications to determine which of the neighboring regions 
are likely to be part of the same object. ARROWs, for example, 
strongly suggest that the two narrow-angle regions belong to the sume 
body (figure 1-4). This evidence is represented by SEE as a strong 
link placed between the regions which are suspected of belonging to 
the same body. Figure 1-5 depicts a link which has been placed 
between regions 1 and 2 as a result of the evidence provided by the 
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naaat «. 

Since links are often generated between regions belonging to 
different bodies, SEE is not satisfied with the evidence represented 
by a single link; two or more links are required to bind two regions 
together into a single nucleus . A nucleus represents a set of regioni 
which will eventually be reported as belonging to a single object in 
the scene. Two or more links between a pair of nuclei will similarly 
result in their conglomeration into a still larger nucleus. The 
nuclei grow Ln this fashion until no two of them can be combined by a 
double link; they are then called maximal nuclei * SEE then uses 
weaker evidence in the form of weak links to further merge some 
nuclei. Finally, special heuristics are employed Co attempt to 
eliminate nuclei consisting of a single region by combining them with 
other nuclei. 

This fairly sophisticated procedure can sort out the regions in 
scenes as complicated as that in figure 1-6, borrowed from Gunman's 
thesis. Twelve objects are reported and the regions of each are 
remembered, 
1.1.2.3 Higher-level analyals 

Winston has developed and implemented procedures which use the 
output of SEE to create network-structured descriptions of scenes. 
These descriptions are used by Winston's programs for comparisons^ 
concept learning, and identification. 
1.2 Possible Improvements to SEE 

Refer again to the flow diagram of figure 1-1» Winston haf 
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noted that all information in the system moves In one direction only, 
as indicated by all of Cho arrows in the diagram pointing from bottom 
to top. lie remarks: "There is ob yet no way a process emt discourse 
with and modify the behavior of any process* acting below it." [2J 
One of the aims of this research is to enable SEE to receive such 
discourse from above. 

Another possible improvement to SEE is indicated by the scene of 
figure 1-7. Vertices a, b, and c each generates one strong and one 
veatc link, causing SEE to report a single body composed of regions 1, 
2, 3, and U. A more satisfactory analysis would find two abutting 
wedges. 

In figure 1*6, which is ambiguous, SEE again finds one body. 
Either of the two alternative analyses depicted would be better. 
1.3 SEEMORE: an extension of SEE 
1*3.1 Basic properties of SEEMORE 

In view of the problems of SEE noted above, it would be useful 
for a modified SEE procedure to fulfill the following requirements: 

1) It should correctly handle all cases which SEE handles 
correctly. 

2) It should correctly handle many cases, such as figure W» 
which SEE handles incorrectly. 

3) It should provide for interaction with a higher-level 
object-recognising program such as Winston's. 

4) In case its first analysis is unsatisfactory, or in case of 
ambiguous scenes such as figure 1-8, Lt should be capable of 
generating plausible alternative answers. 

I have devised a procedure called SEEEORE which seems to satisfy 

all of the above criteria for many scenes. SEEMORE has not been 
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programmcd , but has been developed extensively through hand-simulations 
on a large number of scenes. For the scene of figure 1-6, for example, 
SECttRE produces the same answer as SEE, finding twelve bodies, In 
figure 1-7 * SE£»RE finds the wedges (1 3) and (2 4). Por the 
ambiguous scene of figure 1*8, SE&SORE produces both of the alter- 
natives depicted, 

SEfflORE can also handle the difficult scene depicted In figure 
1-9. In this case, four bodies are reported. These are composed of 
regions (2 3 4), (5 6), (6 9 10), and <1 7 11). 
1.3.2 Splitting 

Besides forming links in the manner of Guzman 1 * program, SEEMORE 
also forms splits . While a link represents evidence that two regions 
are part of the same body, a split represents evidence that two 
regions belong to distinct bodies in the scene. To implement the 
splitting process I have developed a variety of heuristics, similar to 
those of SEE, which utilise simple local evidence to begin splits at 
vertices and extend them along edges of objects. These new heuristics, 
along with a modified version of SEE, are incorporated into a 
procedure which attempts to generate alternative answers, starting 
with the most plausible- 
1.4 A note on nomenclature 

Figure 1-10 Illustrates some of the abbreviations and symbols 
which will be used In the text and diagrams of this paper. Numbers, 
such as 3 or 3, represent regions in the scene. The background, 
which is region 5 In this scene, will often not bo labelled in the 
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reroaining diagrams. Lover-case letters, such as a, bj and c in the 
figure, wilt represent vertices in the scenes. The wavy lines super* 
irt posed on edges cb And ba represent split?* These vill be designated 
in ths text individually as c-b and b-a; or, equlvalently, as one 
continuous split c-b-a. Strong and weak links are designated by short 
solid and dotted lines as shown, following Cusman's usage. Conglomer- 
ated regions will be indicated by enclosing lists of such regions in 
parentheses. In this example, the bodies found are (1 2) and <3 4). 
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2 The basic 5EEM3RE procedure 

Hie flowchart of figure 2-1 illustrates the general struct urr cif 
SEEWRE. The procedure begins by Applying very cautiously those 
heuristics vhlch arc believed to be most tellable; this step It 
referred to at initial splitting and linking, or phase 1, The next 
step in SEEHORE'a analysis of a scene, called phase 2» is a more 
tentative attempt to generate splits by applying heuristics thought to 
be less reliable than those of phase 1. If there Is no splitting 
evidence in the scene, phase 3 is entered directly, and 5EEM0M 
reduces to a slightly modified version of 5EE. 

Finally, phase 3 applies the rest of SEE'i heuristics to the 
tentatively split scene and produces SEEMOHE's first attempted solu- 
tion. This is submitted to the higher-level program for approval. If 
the higher-level program is dissatisfied with the first analysis, it 
will return to SEEMDRE with an appropriate complaint. The processing 
of this complaint is discussed in the next chapter. 

At various times in phases 1 and 2, before the analysis of the 
scene is complete, SEQCRE may consult a higher-level program to find 
out if certain sub-parts of the scene can be recognized as unoccludcd 
objects. This Information is used by SEQDBE to generate additional 
splits and links. 

2.1 Phase 1: generate Initial splits and links 
2.1.1 Split ting- vert ices 

5EDDRE begins by identifying those vertices in the scene which 
may be used to initiate or e«end splits. I refer to these, naturally 
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enough, as spllttinft-vcrtlce* . A coimoonly occurring splitting-vertex 
is the generalised four-line vertex, represented In figure 2-2A. The 
K and X joints are special eases of four-lint vertical. More general* 
ly, any vertex of four or sore line* la a aplitting-vertex. In the 
example of figure 2-3, SEXMOKZ marke vertlcea a, b, c, d, and e aa 
apl 1 1 1 lag- ver t ices . 

Hot ice In figure 2-4A that vertlcea a and b would be classified 
aa splitting-vertices. However, theae are falae vertlcea, in the 
sense that they do not peralat under a email change in viewing angle, 
aa shown In figure 2-4B. On the other hand, vertlcea c and d are 
Invariant under small changes in viewing angle. 5EW)KE assumes that 
all vertlcea In the Input scene are similarly Invariant; scenes like 
figure 2-4A will, in general, not be analyzed properly. 

Any split which nanates from a vertex v is terminated at the 
next vertex encountered on the split line, unleaa the next vertex la 
an L. In figure 2-5, for example, a alnglc split initiated at vertex 
v along the line v-a is extended through vertex a and terminates at 
vertex b aa ahown. 

Aa illustrated In figure 2-6, a split propagates through a pair 
of matching-Ts if the same region R borders on the center lines of 
both Ta. 
2.1.2 Phase I heuristics 

Ihe phase 1 heuriatlca are uaed to Initiate apllta and links 

■ 

based upon evidence provided by splitting-vertices and special 
configurations as described below. These heuristics are considered 
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to be more reliable than thoae which will be applied later In phase C. 
2.1.2.1 Heuristic: split around strongly-conglomerated object? 

In order to implement this heuristic, SKKMGRK first Invokes an 
initial-linking routine. Thcpc initial links are placed much more 
cautiously than the more general SEE-type links to be used later* 
Vertices at which initial links art placed are illustrated in figure 
2-7. These links are subject to the veto of a link-inhibitor of the 
sort used In SEE. Situations in which links will be inhibited are 
illustrated In figure 2-8. A more lenient link-inhibitor, to be 
described later, will be used to complete the linking in phases 2 and 

3. 

Any two regions joined by at least two initial links are said to 
be members of the same at rongly- consigner a ted nucleus * A region is 
considered to be a member of such a nucleus only If It is doubly 
linked to another single region within the nucleus; single links to 
two different regions in the nucleus do not suffice. Each strongly- 
conglomerated nucleus of tw or more regions found in the scene is 
submitted to a higher-level program capable of recognizing objects. 
The higher-level program should verify the nucleus only if it makes 
asnse as a complete unoccluded object In the scene. A nucleus so 
verified is called a strongly-conglomerated object . ^ 

The use of these objects in splitting Is Illustrated in figure 
2-9. The general splitting rule Is shown In figure 2-9A. Strongly* 
conglomerated object (1 2 3) has a splitting-vertex v on its boundary; 
splits are made which extend from vertex v to neighboring vertices 
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a and b along the boundary of object (12 3). For example, in figure 
2-9B, split c-d-e is caade along the boundary of strongly-conglomerated 
obj<ct (4 5 6). It should be noted that the phase 2 *plit*«Xl«rtinfl 
heuristics, to be described later, arc n ot applied to the splits just 
described. 

2.1.2.2 Heuristic: internal-T 

When a T-joint borders on background as shown in figure 2*10, a 
split is placed on the center line of the T. 

2.1.2.3 Heuristic: special KULTI 

At ahown in figure 2-11, if a vertex has four or more lines of 
vhich tvo are collinear, and of which only one falls to one side of 
the collinear lines, then a split is placed along that single line. 
In the special case of a four-line vertex, this heuristic reduces to 
the X-joint heuristic illustrated in figure 2-12. In this ease, if 
either of the links shown in figure 2-12 has been inhibited during the 
initial linking step, then the splits are inhibited as well. 

2.1.2.4 Heuristic: split to external concavities 

This heuristic is best explained by an cxaaiple. In figure 2-13, 
the FOHK vertex a borders on background; the line vhich does not 
border the background connects to splitting-vertex £* In this case > a 
split is made along v-a. Vertex a Is referred to as an external 
concavity. 

More generally, this heuristic places splits from splitting- 
vertex v to external concavity a in either of the situations depicted 
in figure 2-14. 
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It is conjectured that It would also be useful to split between 
FORKa and external concavities; In other words, to allow the vertex v 
In figure 2-14 to be a FORK. This additional heuristic would place the 
Important split a-b In figure 2-15, enabling body (1 2) to be 
separated from region 3. However, this heuristic is not included In 
the current version of SEEWDRE pending further study. 
2.1.2.5 Heuristic: split the crossbar of the T-AKROW 

The configuration for which this heuristic applies is illustrated 
in figure 2-16. Splits propagate along the crossbar of the T in both 
directions. In figure 2-17, this heuristic generates the split a-b-c. 
A variation of this heuristic which Is employed in phase 1 is the T-X 
configuration depleted in figure 2-18. Under study but not being used 
is the T-ilnk configuration shovn in figure 2-19. 

2*2 Phase 2: Generate other plausible splits 
2.2.1 Extending splits 

This routine uses evidence provided by splitting-vertices and 
other configurations to extend splits which have been placed in phase 
1. The routine applies only a couple of heuristics at present, and 
may represent an area in which new heuristics and some fine tuning 
would help significantly the performance of the 5EEM0RE procedure 
on difficult scenes. 

A very coooon situation for extending a split Is represented In 
figure 2-20. If exactly one split terminates at a four-line vertex, 
such as split a-b In the figure, then the apllt is extended along the 
opposite line b-e. In addition, links arc placed across the two 
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unsplit line* as shown. Figure 2-21A represents a scene in which in- 
itial splitting and linking has been completed, and the split- 
extending heuristic applies. The result Is shown in figure 2-21B. 
Notice that the link at vertex c has been broken by the split from b 
to e. The split-extending would be finished at this point, since no 
other splitting-vertices remain. 

This split-extending heuristic applies to Ks and Xs as special 
cases of four-linr vertices, but docs not apply to pEAKs, 

Another, rather specialited, split-extending heuristic applies 
to the five-line HULT1 with two eoliinear lines, illustrated in 
fipurt 2-22. Figure 2-23 shava an nmpU of this configuration In 
a ncene. 

If either of the links generated by one of the above heuristics 
is inhibited by the link-inhibitor described below in the discussion 
of phase 3, then the split is not extended. The other link is still 
placed if not Inhibited. 

Finally, another specialized heuristic is depicted in figure 
2-74A. Although FORK b and ARROW d have both linked regions 1 and 2 
in the initial-linking phase, the split entering the FORK along a-b 
provides strong enough evidence to break both the links by extending 
the split along b-d. Of course, the symmetrical split along b-c 
would also have been sufficient evidence, A split is similarly 
extended from the FORK to a PEAK as in figure 2-24B. Applied to the 
ambiguous scene shown in figure 2-25A, this heuristic enables SEtfMQRK 
to produce the analysis represented in figure 2-25B. 
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The split-extending routine Iterates until no further extending 
can be done. If any strongly-linked nuclei ire formed during split 
extension or at any subsequent point in phase 2, the routines which 
were employed at the beginning of phase l (section 2,1.2.1) are 
invoked to verify objects and make splits accordingly. 
2.2.2 Apply weaker splitting heuristics 

After the initial splits of phase 1 have been extended, there 
often remain splita which are incomplete or splitting-vertices through 
which no splits have yet traversed. If this is the case, SEEMORH 
employs weaker and more conditional evidence Co propose and extend 
additional splits. As will be seen later, SEEMORH stands ready to 
generate alternative analyses) if requested by a higher-level program, 
by backing up and placing alternative splits. 

The remainder of this section catalogues several configurations 
which seem to provide weaker evidence for splitting in many scenes. 
SEEHORE attempts to apply these heuristics in the order In which they 
are presented below. The first applicable rule is invoked in each 
iteration until the list is exhausted; splitting is then complete, 
2.2.2.1 Heuristic: split along the non-adjacent lines 

This is a generalization of the split-extending rule for four- 
line vertices. Given the generalized four-line vertex of figure 2-26, 
this rule indicates the alternatives of splitting along either a*b*c 
or d-b-e. 5EEM3RE takes these alternatives into consideration when 
applying any of the heuristics below to a four-line vertex. 
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2+3+2+2 Heuristic: sake the most complete split possible 

The above title is actually a general statement of two specific 
rules which are applied at each four-line vertex which is not a PEAK, 
has not yet received any splits, and does not border on the background. 
One of these rules is: extend an unfinished split to the background or 
to another unfinished split whenever possible. For this purpose, an 
unfinished split is defined as one which terminates at a vertex not 
bordering on the background. Additionally, if a split traverses a 
MULT I but leaves three or more adjacent lines unsplit as shown in 
figure 2-27 > an unfinished split is considered to exist at the MULT I. 
Look at figure 2-28- Notice that an initial split is made along c-d ( 
then extended along d-c. The present rule favors splitting c-a-b 
rather than g-a-f, in order to complete the unfinished split ending at 
vortex £■ 

The other specific rule, to be tried if the first does not apply, 
is: split to the background whenever possible. In figure 2-29, for 
example, we consider splitting either b-a-c or e-a-d. The present rule 
would prefer the former split, since b and £ both border on the back- 
ground, while £ is an internal vertex. 

In the example of figure 2-30, the spl U-to-background rule 
indicates splits c*a-d and c-b-d. The resulting analysis proposes 
objects (1 2), (3 4), and (5 6), which could plausibly represent three 
tetrahedra. Huaans seem to prefer the analysis (1 3 5) and (2 4 6), 
which is produced as an alternative by SEEMORE. Perhaps the appro- 
priate heuristic here involves minimizing the length of splits used or 



-39- 




Figure 2-28 




Figure 2-29 




Figure 2-30 



-40- 



the number of bodies found; neither of thi.'se factors is taken into 
account by SEDtORE. 

One further example indicates the power of the most-complete-split 
heuristic. In figure 2-31, SEEMORE uses this heuristic to correctly 
split along e-d-a, producing the plausible bodies (I 2 3) p (4 6), and 
(5 7). The inherent complexity of this scene Is reflected in SEE'* 
answer; bodies (1 2 3), (4 5 7), and (6) are reported, 

2.2.2.3 Heuristic: split becween a pair of splitting-vertices 

This rule applies only to pairs of splitting-vertices which are 
neighbors on the same line (figure 2-32) or which have a common 
neighbor (figure 2-33). Each of the vertices £ and b in the figures 
must be either a PEAK, a KULTI of more than four lines, or a four-line 
vertex which has thusfar received no splits. Furthermore, a and b 
must be neighboring only each other in this fashion. 

If these conditions are met, splits are placed as shown in 
figures 2-32 and 2-33- The split-extender is used if applicable. 

Both forms of this heuristic may simultaneously apply in some 
cases. An example is figure 2-34, which contains two neighboring 
PEAKS. The first analysis splits between the PEAK* directly, along 
*-b. This wilt generate SEEttOFE's initial guess; bodies (1 2 3) and 
<4) are reported (figure 2-35). If an alternative is requested, the 
alternative split to a common neighbor of the pair of splitting- 
vertices is used. In this case, the only common neighbor vertex is c, 
and the j»U*rn^riv* split i* a*c-b, Tbe result f.ng alternative answer 
reports bodies (1), (2), and (3 4), as shown In figure 2-36. Both 




-41- 



Fi&ure 2-31 




Figure 2-32 




Figure 2-33 



-42- 




» 



Figure 2-3^ 




Figure 2-35 



Figure 2-36 




-43- 



thcsc alternatives arc plausible analyses of the scene, 

2.3 Phase 3: link and conglomerate 

When phase 2 of SEEHORE cannot start any new splits or extend any 
existing ones, phase 3 is entered. Phase 3 applies the remaining 
heuristics of SEE* with a few modifications as outlined below. In 
this phase the remaining links are placed, subject to the constraints 
of splits already made. Nuclei are formed and single-region bodies 
are handled in similar fashion to SEE. Phase 3 generates a tentative 
analyfis, which is presented to a higher-level progran for approval. 
2*3,1 Completion of linking 

3he initial linking has already placed links at FOBXs, AKKOWs, and 
X*Jolnts as shown in figure 2-37. Some of these will have been 
inhibited by the phase 1 link-inhibitor. Others will have sub- 
sequently been broken by splits. Kew links may have been added by the 
phase 2 routines. The linking is now completed by placing strong and 
weak links exactly as SEE does* except as specified in the paragraphs 
belov. Refer to Guzman's Ph.D. thesis [4], pages 81 through 88, for 
detailed descriptions of the linking heuristics. 
2.3.1.1 Inhibition of links 

Figures 2-38 and 2-39 illustrate the situations in which links 
are inhibited during the final linking stage of SEEMORE. Figure 2-38 
depicts inhibited links which are also inhibited by SEE. These 
include links across lines with passing-Ts; notice, however, that 
ARROW links are not inhibited by the passlng-T. Also inhibited are 
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links between any object and background; and links across the outside 
lino of a PEAK or an ARROW. Notice that the through-Ts function has 
been restricted as shown in figure 2-39. In the matcMng-T situation 
»arked in the figure by a broad arrow, the same region R must border 
the center lines of both of the T-Joints. R In this case may not b<t 
background . 

Links arc also inhibited across split lines (figure 2-40)* Also 
shown in figure 2-40 is the pair of neighboring FORKS* across which 
strong links are inhibited and a single weak link is placed as shown. 

A linking heuristic used in Guzman's program is illustrated In 
figure 2-41A. It is omitted from SEEMORE because it is inconsistert 
with the assumption I have made that the type of each vertex is In- 
variant under small perturbations of the viewing angle. Another 
heuristic used by SEE, illustrated in figure 2-AlBi is made unneces 
sary by other heuristics of SEEMORE. 

Unlike SHE, SEEMORE does not place matching-! links at this step. 
These are treated separately, as discussed below, 

2.3.1.2 Formation of nuclei 

On the basis of strong and weak links which have been placed up 
to this point, SEEMORE conglomerates nuclei using the heuristics of 
Guzman's program. After conglomeration is complete, matchlng-T links 
are processed as described in the next section. 

2.3.1.3 Hatching-Ts 

SEEMORE's decision to link or not to link across a pair of 
matching-Ts is always made provisionally. If the higher-level program 
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registers an appropriate complaint, SEEMORE stands ready to change its 
mind about the matchtng-T links involved. 

Figure 2-42 illustrates two possible matching-T configurations. 
In figure 2-42A, the same region R borders on the center line* of both 
Ts. In this*case, SEEMORE will place a link between regions 1 and 2. 
This heuristic still applies if R is background. Kotlce that If 
regions 1 and 2 were also identical, there would be no need to plate 
any links at all. The link between regions I and 2 will not be placed 
if there is a split between any member of the nucleus to which region 
1 belongs and any member of the nucleus to which region 2 belongs. 

In figure 2-42B, regions 1, 2, 3, and 4 are all distinct. In 
this case SEEMORE does not place links on the first pass, but plan;; 
to link 1 to 2 and 3 to 4 as an alternative. As before, matching-T 
links between two different nuclei are inhibited if a split exists 
between members of those nuclei. 

Figure 2-43 illustrates some typical applications. When presen- 
ted with the scene of figure 2-43A, SEEMORE generates a split and 
links as shown. The split inhibits all of the possible matching-T 
links; SEEMORE finds bodies (4 6 7), (1 2 3), and (5 6). In figure 
2-43B there are no splits. Using the matching-! heuristic, SEEMORE 
places a single strong link between regions 4 and 5 P and another 
single link between regions 7 and 8. These arc sufficient to con- 
gloneraie body (4567 6). If the higher-level program indicates 
that (45678) must be split apart, SEEMORE will back up and remove 
one or both of the offending links, depending on the exact form of the 
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complaint. The result will be two separate bodies, (A 6 7) and (5 6). 

The scene of figure 2-43C would present problems for SEE due to 
the large number of spurious matching-T links- Using the modified 
matching-T heuristic, SEEMORE does not place any of the spurious links; 
all of the separate bricks arc correctly identified. 
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3 Producing alternative answers 
3*1 The nature of a complaint 

If a higher-level procedure tt dissatisfied with SEEMORE's fiist 
attempt, it should ideally return to SEEMORE with the following inior- 
mat ion; 

1) a list of the regions which have not been satisfactorily 
conglomerated 

2) the nature of the complaint: body has coo many regions/ 
is overlinked; body has too few regions/ is underlined; 
body (or bodies) unrecognizable / does not make sense; 
analysis of the scene is plausible, but alternat ive(*) 
desired* Specific regions should be related to each 
complaint. 

Of course, it may not always be possible for a higher-level 
program to provide a complaint in such detail. It is conceivable chat 
a number of objects, or even Che whole scene might be Incomprehensible 
to -he higher procedure. If there are not many splitting vertices in 
the scene, the number of alternatives which SEEHORE can generate is 
email and could be exhaustively enumerated if necessary. Usually, 
among these alternatives would be the original SEE' s analysis of the 
scene. 
3.2 The alternative-list 

Each time a splitting heuristic is employed in phase 1 or pha«e 2 
of SED40RE, a notation is made at the top of a special list called the 
alternative-list . Each entry in the alternative-list identifies the 
regions for which the given heuristic has made a splitting decision, 
called reference-regions, and indicates what step(s) should be taken 
to generate an alternative splitting analysis for the specified 
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regions. The alternative-list entry also indicates what sort o£ 
change can be expected if the given alternative is employed: 
specifically, the entry may indicate more-splitting, less-splitting* 
or different-splitting. A more-splitting alternative is expected to 
split apart two or more currently-conglomerated regions; a less- 
splitting alternative is likely to conglomerate currently separated 
regions; while a different-splitting alternative will help to gener- 
ate an alternative partitioning of the IndLcated regions. The 
splitting and linking which is indicated on the alternative- list la 
described, but not actually carried out unless a complaint is being 
acted upon. 

Kotiee that the alterftative-lUt ifl erdered according ta a 
heuristic measure of strength, with alternatives produced by less 
reliable rules appearing nearer to the top of the list* Later, when 
processing a complaint, SEEMORE will try these alternatives in order 
starting from the top of the list, since the alternatives appearin£ 
there were produced by heuristics more likely to make mistakes. 

3.2.1 Producing alternative-list entries -- phase 1 

There is only one alternative to applying a given splitting 
heuxistic in phase 1: not applying It. This will be entered at the 
top of the alternative-list a^s a less-splitting alternative, along 
with the regions between which the given split passes. 

3.2.2 Producing alternative-list entries *- phase 2 

Let us first consider splits which are extended by one of the 
split-extending heuristics. In all such cases, the alternative is 
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not :o extend Che split. This is entered on the alternative-list a* 

i 

a less-splitting alternative* with the bordering regions of the 
extension split as reference-regions. 

For any split generated in phase 2 by the most-completc-split 
heuTistic, two entries are made in the alternative-list, Tnc first of 
these, a less-splitting alternative, inhibits the split entirely. The 
other entry, representing a different-splitting alternative, indicates 
both omitting the original split, and placing an alternative split 
along the pair of lines originally unsplit. 

Figure 3-1 illustrates the rule just stated for a four-line 
vertex. In figure 3-1A, the most-complete-spltt heuristic has been 
applied to vertex * to produce the splits and links shown. At the 
same time, tvo alternatives are indicated on the alternative-list. 
The less-splitting alternative, with no splits or links, is shown in 
figure 3-lB. The different-splitting alternative is depicted in 
figure 3-1C. Here the original split has been abandoned, while 
alternative split c*a-b is made and links are produced as shown. 

If the split-between pairs-of-splitting-vertices heuristic is 
applied, one indicated alternative is to split to a common neighbor 
vertex, as illustrated in figure 3-2. This is a different-splitting 
alternative. Additionally, the alternative of no split at all is 
indicated as a less-splitting alternative. 

Throughout phases 1 and 2, whenever a split Is inhibited, a more- 
splitting alternative is recorded which makes the inhibited split. 
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3«3 The processing of a complaint 

When a complaint is Issued by a higher-level program, SEFMORE 
expect* to be provided, as mentioned earlier, with a list of regions, 
and >:he nature of the complaint associated with the given regions. 
Starting at the top of the alternative-list, SEEHORE examines each 
entry in the list until it finds one whose reference-regions match 
well with the list of troublesome regions, and which is of a type which 
would tend to correct the complaint. For example, if the complaint 
says that a body has too many regions, SEETORE will look for more- 
splitting entrlea with reference-regions from the given body. If an 
appropriate entry is found, SEEHORE takes the steps indicated in the 
entiyt Then the phase 3 routines are used to complete the linking and 
see if an alternative analysis has actually been generated. If thvrc 
are no more- $p\ it t ins entries in the alternative-list with appro- 
priate reference-vertices, or if no alternative is actually produced 
by the entries considered, then SEEHORE makes one last effort by 
searching for different -split ting alternatives with the proper 
reference-vertices. If no alternatives are ultimately produced, 
SEEHORE would have to report its failure to generate an alternative 
analysis. If an alternative is produced, it is submitted to the 
higher-level program for approval. Additional alternatives could be 
produced in the same manner if necessary. 

If the complaint had been that a body was composed of too few 
regions, the above steps would be followed, except that less-splitting 
alternatives would be sought instead of more-splitting ones. 
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If the higher- level program found the original analysis plausible, 
but desired an alternative if possible* then dtf ferent-splitt tnc 
alternatives would be sought first, followed by more-splitting and 
lees-splitting alternatives if necessary. 

Finally, if the higher-level program complained that it could 
make no sense of SEEMORE's analysis, any type of alternative would be 
considered. In this case, it is of great help to SEEJttRE If the 
higher-level program specifies regions which it suspects of causing 
difficulty, 

SEEMORE's alternative-generating procedure is successful in 
producing the alternatives shown for the simple ambiguous scene of 
figure 3-3, and the oore complicated scene of figure 3-4. 
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4 Some examples 
4.1 A detailed example 

I can now explain in detail Che operations performed by SEEMORE 
In analyzing the example presented earlier, which is reproduced in 
figure 4-1. 

liinks made by the initial linking routine are depicted in figure 
4-2A. Nuclei (2 3) and (9 10} are proposed as unoccluded bodies which, 
presumably, are rejected by the higher-level procedure. Figure 4-2B 
shows the link diagram at this point- As shown in figure 4-3, the 
initial splitting routine initiates splits along a-f» b-c» and j-e. 
This completes the initial splitting and linking phase <phasc 1). 

In the additionai-iplits phase (phase 2) t further splitting 
operations arc performed. First, the split along b-c is extended to 
vertex h» which also links regions (5 6) and (9 8) at vertex £. 
Similarly, split a-f is extended to £> and split J-e is extended along 
t-g. Then split h-d-k is generated at splitting-vertex d by the most- 
complete-split heuristic. The split h-d-k also produces the links 
(7 11) and (8 10) at vertex d. Additional-splits phase 2 has now 
been completed, as illustrated in figure 4-4. 

Finally, phase 3 is invoked- The remaining strong and weak links 
are placed, as illustrated in figure 4-5- The corresponding link 
diagram is shown in figure 4-6. Next, procedures similar to the rou- 
tines of SEE are used to conglomerate regions on the basis of the 
strong and weak links which have been placed. The final nuclei are 
shown in figure 4-7. SEEMORE reports the four bodies (2 3 4), (5 6) » 
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(8 9 10), and (1 7 11). 

* 

4,2 Other examples 

Figures 4-8 through 4-12 present several additional examples la 
Abbreviated form. For each example the scene is presented with all 
splits and links indicated, and the bodies found by SEEHORE are listed. 
In the examples of figures 4-9 and 4-12, an alternative analysis 
generated by SEEHORE is also shown. 
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5 Failures of SEQORE and recommendation* 

SEEWORB often relics heavily for success upon the technique of 
congloaerat ing and recognizing unobscured objects early in its 
analysis. For example, In figure 5-lj SEEMORE correctly splits along 
c-a-b-e by first verifying object (1 2), Since the corresponding 
object in figure 5-2 is not fully visible* splits c-a-d and e-b-d will 
be generated. Consequently, SEQ40RE reports objects (1 2 3), (4 6), 
and (7 5 8 9). Notice that figure 5-2 is a difficultscene for 
human*, too. 

Although SEEMORE does provide for interaction vith a higher* 
level routine, perhaps this interaction has not been carried far 
enough. For example, an extended version of SEEHOHE might be a van.* 
of the kind of body it was looking for, say a rectilinear brick. It 
might recognize a FORK as the possible corner of a brick, then extend 
the lines of the FORK looking for the other corners. Or perhaps, 
having recognized an object in the scene, It would be able to 
hypothesize what the scene would look like if that object were 
removed. It could then proceed to analyze the simpler scene, pro- 
cedures with such capabilities could only result fron a major over- 
haul of the present vision system. 

Another kind of problem results from certain scenes in which 
crucial splitting evidence is obscured. An example of such a scene is 
figure 5-3. What is needed here is a line-extender, whose use might 
be indicated by the T-joints at ^, b, £» and £. 
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